The present invention relates generally to storage systems and, more particularly, to methods and apparatus for managing the data location at a storage system, especially a storage system that has the chunk allocation capability to allocate a chunk to a logical volume from a chunk pool as well as the capability to access files in internal or external file systems.
To reduce wasted physical blocks in raw storage media, “thin provisioning” based on dynamic chunk allocation capability is adapted to storage system configurations. This type of storage system has special logical volumes, which are logical volumes that have no physical blocks initially. The storage system constructs a chunk pool with raw block devices and allocates a chunk from the chunk pool when the logical volumes receive a write command and a chunk has not yet been allocated to appropriate physical blocks as specified in the write command. Using this technology, the user of the storage system does not need to prepare many raw block devices or large raw block devices initially and can add devices later as more volumes are actually consumed.
When using the thin provisioning technology, the chunk size is an important factor for the effectiveness of reducing wasted blocks. A whole chunk is allocated even when the host computer writes only a few bytes onto the un-allocated area. Thus, if the chunk size is large, a large volume of raw block devices is unnecessarily consumed. Furthermore, the deduplication method between chunks will not work effectively because it is less probable that larger chunks have the same contents. By making the chunk size smaller, the effectiveness of reducing the waste and the de-duplication will be better. However, smaller chunks require more memory for the chunk mapping table between data volumes and raw block devices.
U.S. Pat. No. 6,725,328 discloses the dynamic chunk allocation method and apparatus to realize thin provisioning storage. U.S. Patent Publication No. 2007/0055713 discloses the dynamic chunk allocation with block devices that have different characters to realize per-chunk tiered storage. U.S. Patent Publication No. 2009/0132619 discloses the deduplication method for a storage system. U.S. Pat. No. 7,330,950 discloses the per-file hierarchical storage management method. The entire contents of these references are incorporated herein by reference.